Diverse Dialogue Generation by Deep Learning Methods Using Loss Function IncorporatingWord Statistics

In recent years, there has been a lot of research on building dialogue systems using deep learning, which can generate relatively fluent response sentences to user utterances. Nevertheless, they tend produce responses that are not diverse and less context-dependent. Assuming the problem is caused by Softmax Cross- Entropy (SCE) loss, treats all words equally without considering imbalance in training data, loss function Inverse Token Frequency (ITF) multiplies SCE weight based inverse token frequency, was proposed confirmed improvement diversity. However, diversity sentences, it necessary consider only information independent tokens, but also frequency incorporating sequence tokens. Using frequencies incorporate tokens compute weights dynamically change depending context, we better represent seek. Therefore, propose function, N-gram (INF) weighted n-gram instead order confirm effectiveness method INF conducted metric-based human evaluations automatically generated models trained Japanese English Twitter datasets. evaluation, Perplexity, BLEU, DIST-N, ROUGE, length were used as evaluation indices. assessed coherence sentences. model achieved higher scores ROUGE than previous methods. showed superior values.